Skip to content

Add SGL support for IB rc, cuda copy and cuda IPC (imported)#11637

Open
bosilca wants to merge 9 commits into
openucx:masterfrom
bosilca:topic/add_uct_ib_cuda_SGL
Open

Add SGL support for IB rc, cuda copy and cuda IPC (imported)#11637
bosilca wants to merge 9 commits into
openucx:masterfrom
bosilca:topic/add_uct_ib_cuda_SGL

Conversation

@bosilca

@bosilca bosilca commented Jul 8, 2026

Copy link
Copy Markdown

Adds native UCT transport support for the scatter-gather list (SGL) zero-copy put operation (uct_ep_put_sgl_zcopy), which previously only had the UCP software-emulation path. This lets a multi-segment ucp_put_nbx SGL operation be offloaded directly to the transport as a single batched submission instead of being decomposed into individual puts.

Implemented for:

  • IB RC — rc_verbs (chained ibv_send_wr batch, single doorbell) and rc_mlx5 (one WQE per segment, batched).
  • CUDA — cuda_copy (intra-process device↔device) and cuda_ipc (inter-process device memory).
    Also adds gtest coverage and a ucx_perftest bandwidth mode for benchmarking.

Details / commits

  • BUILD: stop treating -Wunknown-pragmas as an error so compilers without OpenMP #pragma omp support can build.
    UCT/IB/RC (base): advertise the max SGL segment count (bounded by QP depth), add a v2 capability-query helper, and reserve the full TX credit count for a multi-WQE SGL op so it is queued as pending instead of busy-spinning when credits are momentarily short.
  • UCT/IB/RC/VERBS: implement uct_rc_verbs_ep_put_sgl_zcopy, posting the whole segment list as a chained RDMA_WRITE batch with one doorbell ring.
  • UCT/IB/MLX5: implement uct_rc_mlx5_base_ep_put_sgl_zcopy; add an iov_post_flags argument to uct_rc_mlx5_txqp_dptr_post_iov to skip forced TX moderation on inner batched WQEs (DC caller updated for the new signature).
  • UCT/CUDA/COPY: implement uct_cuda_copy_ep_put_sgl_zcopy and use a process-wide interface id so cuda_copy interfaces in the same process are mutually reachable and can serve as the SGL RMA lane.
  • UCT/CUDA/IPC: implement uct_cuda_ipc_ep_put_sgl_zcopy (originates from michal-shalev's uct-cuda-ipc-sgl branch).
  • TEST/GTEST: enable the UCP SGL put tests and adapt the CUDA IPC device SGL test to the new API.
  • TOOLS/PERF: add -D sgl[/N] to ucx_perftest (ucp_put_bw), splitting the message into N equal segments and driving it through the SGL put API.

Notes

  • The counts/strides (strided/repeated) SGL fields are handled where the transport supports them; the basic N→N mapping is the primary path.
  • '-D' sgl is restricted to the UCP put bandwidth test.

Tests

  • gtest: test_ucp_rma_sgl.* (host and CUDA) and uct/cuda/test_cuda_ipc_device SGL cases pass.
  • ucx_perftest -t ucp_put_bw -D sgl/4 ... runs over rc_verbs, rc_mlx5, and intra-node cuda_copy/cuda_ipc.
  • Size sweep vs -D contig shows expected bandwidth.
  • Build passes on a compiler without OpenMP pragmas.

Follow-up

  • while verifying the rkey handling I noticed two now-redundant uct_ib_md_direct_rkey() calls in the RC put paths. Both the mlx5 and verbs put flows already resolve the remote key in place via their fence_put helpers — uct_rc_mlx5_ep_fence_put() and uct_rc_ep_fence_put() set *rkey = uct_ib_md_direct_rkey(*rkey) (or the atomic-resolve variant) before the key is ever posted. As a result, (1) the uct_ib_md_direct_rkey(rdma_rkey) inside uct_rc_mlx5_base_ep_zcopy_post() (rc_mlx5.inl) and (2) the uct_ib_md_direct_rkey(rkey) in the verbs SGL fill (UCT_RC_VERBS_FILL_RDMA_WR in uct_rc_verbs_ep_put_sgl_zcopy) both re-apply the conversion to an already-resolved key. Since uct_ib_md_direct_rkey() is just a low-32-bit mask ((uint32_t)uct_rkey), re-applying it to a value whose high bits are already cleared is idempotent — a no-op — so these calls are harmless but redundant. (On mlx5 it's doubly redundant, because uct_ib_mlx5_ep_set_rdma_seg() only writes the low 32 bits of the key into the WQE regardless.) They can be dropped for clarity, or kept for visual symmetry; either way there is no behavioral difference.

@bosilca bosilca requested a review from michal-shalev July 8, 2026 17:03
@bosilca bosilca force-pushed the topic/add_uct_ib_cuda_SGL branch from f919e9e to a7edbe4 Compare July 8, 2026 19:22
bosilca added 9 commits July 8, 2026 22:25
Downgrade -Werror=unknown-pragmas to a warning so compilers that do not
support OpenMP '#pragma omp' can still build the tree.
Add the shared RC foundation for scatter-gather put_sgl_zcopy: advertise
the maximum number of segments (bounded by the QP depth), expose a
capability query helper, and record the required TX credit count so a
multi-WQE SGL operation is queued as pending instead of busy-spinning
when credits are temporarily unavailable.
Implement uct_rc_verbs_ep_put_sgl_zcopy, posting the whole segment list
as a chained batch of RDMA_WRITE work requests with a single doorbell
ring, and wire the operation and its v2 capability into the interface.
Implement uct_rc_mlx5_base_ep_put_sgl_zcopy, posting one single-SGE
RDMA_WRITE WQE per segment as a batch. Add an iov_post_flags argument to
uct_rc_mlx5_txqp_dptr_post_iov to skip forced TX moderation on the inner
batched WQEs, and wire the operation and its v2 capability into the
interface. The dc_mlx5 caller is updated for the new post_iov signature.
Implement uct_cuda_copy_ep_put_sgl_zcopy for intra-process device copies
and use a process-wide interface id so cuda_copy interfaces within the same
process are mutually reachable and can serve as the SGL RMA lane.
Implement uct_cuda_ipc_ep_put_sgl_zcopy for inter-process device memory
transfers over the SGL put API.

This code originates from michal-shalev's uct-cuda-ipc-sgl branch.
Enable the UCP SGL put tests and adapt the CUDA IPC device SGL test to the
put_sgl_zcopy API (counts/strides), moving capability checks after entity
initialization and skipping when CUDA memory detection is unavailable.
Add a '-D sgl[/N]' datatype to ucx_perftest that splits the message into N
equal-length segments and drives ucp_put_bw through the SGL put API. The
SGL state is heap-allocated to avoid inflating the runner stack frame, and
the option is restricted to the UCP put bandwidth path.
Signed-off-by: George Bosilca <gbosilca@nvidia.com>
@bosilca bosilca force-pushed the topic/add_uct_ib_cuda_SGL branch from a7edbe4 to 4d03b64 Compare July 8, 2026 19:25
@michal-shalev

Copy link
Copy Markdown
Contributor

Hi George, thanks for posting this PR.
Just to clarify, this overlaps with the SGL work that has already been implemented and is currently being upstreamed.

@brminich brminich left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conflicts with other SGL Prs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants